The present invention is related generally to CMOS imaging sensors, and in particular to an improved CMOS imaging sensor having mixed analog and digital pixel readout for high dynamic range.
Solid state image sensors (“imagers”) are important in a wide variety of applications including professional and consumer video and still image photography, remote surveillance for security and safety, astronomy and machine vision. Imagers that are sensitive to non-visible radiation, for example infrared radiation, are used in some other applications including night vision, camouflage detection, non-visible astronomy, art conservation, medical diagnosis, ice detection (as on roads and aircraft), and pharmaceutical manufacturing.
An image sensor comprises a two-dimensional array of photosensitive elements (pixels) in combination with control and readout circuitry. The pixels are sensitive to incoming radiation. The control and readout circuitry scans and quantitatively evaluates the outputs from the pixels and processes them into an image.
FIG. 1 is a schematic block diagram and approximate physical layout of a typical conventional CMOS silicon imager. The imager comprises an n row by an m column array of pixels implemented advantageously on a single silicon die. Each pixel contains a photo-detector plus control and multiplexing circuitry. An active pixel is a pixel which also includes signal amplification and processing circuitry. Each pixel generates an output signal that is proportional to the accumulated radiation incident on the photo-detector during a defined integration period.
All the pixels in a single row are controlled by a set of row signals generated by a row multiplexer. The row multiplexer contains circuits that perform row address and timing functions within the pixel including pixel reset and controlling the length of the integration period. All pixels in a single row output onto their respective column bus at the same time, but pixels in different rows can output at different times. This staggering allows the pixels in a column to share the column bus, multiplexing their output signals sequentially onto the column bus one row at a time.
All the pixels in a single column send their output signals to a column multiplexer via the column bus. The pixel output signals are multiplexed onto the column bus in response to control signals from the row multiplexer. Circuits within the column multiplexer can perform a number of functions including amplification, noise reduction and multiplexing into predefined video or image formats, e.g. a standard TV video sequence. The video or image signals generated by the column multiplexer can be further processed by an on-chip image signal processor to reorganize, improve and enhance the image.
FIG. 2 is a circuit schematic of a typical conventional CMOS active pixel, commonly known as a 3-T cell. The pixel comprises a photo-detector, an integration capacitor Cint, a source follower device M1, a pre-charge device M2, and a row-select device M3. The integration capacitor may simply be the parasitic capacitance of the photo-detector and M1. The active pixel is controlled by two row signals, pre-charge and row-select. It also connects to the column output bus which is terminated in the column multiplexer with a current source or other suitable load device.
At the beginning of an integration cycle, a pulse on the pre-charge line charges the integration capacitor to a known value via M2. During the integration period, photocurrent generated by the photo-detector in response to incident radiation discharges the integration capacitor. This causes the voltage Vs on the gate of M1 to change. The change in voltage ΔVs is a function of the accumulated photo-charge ΔQ and the integration capacitance Cint according to ΔVs=ΔQ/Cint. The ratio of output voltage to accumulated charge ΔVs/ΔQ=1/Cint is known as the conversion gain. At the end of the integration period, the row-select line is set to allow the voltage Vs to be read out on to the column output bus via M1 and M3. The operation of this type of pixel is well understood by those skilled in the art.
For a given integration period, the minimum optical signal that a pixel can detect is limited by shot noise in the photo-detector, reset noise in the integration capacitor (sometimes known as kTC noise) and electrical noise in the read-out circuitry. The maximum optical signal that a pixel can detect is limited by the charge accumulation capacity of the integration capacitor. Once this limit is reached, the pixel is said to be saturated. The dynamic range of a pixel, typically measured in dB, is the ratio of the maximum optical signal (at saturation) to the minimum optical signal (limited by noise). The dynamic range of a pixel is a measure of the imager's ability to capture both very bright and very dark objects in a single image.
The dynamic range of an imager is the ratio of the maximum optical signal that can detected without pixel saturation to the minimum optical signal that can be detected, allowing for changes in integration time and aperture. The dynamic range of an imager can be much larger than the dynamic range of the pixel. Note, however, that modifications to operation of the imager such as integration time and aperture affect all of the pixels in an imager equally. They allow an imager to operate in very bright light or under very low light conditions. They do not, however, improve an imager's ability to capture very bright and very dark objects in the same scene.
There are a number of applications of CMOS imagers that require very high dynamic range within a single scene. An example is an automotive night vision camera in which a scene to be processed may include both very dark objects (e.g. animal or pedestrian on the road at night) and very bright objects (oncoming car headlamps). Another example is a security camera which is used to identify a poorly lit person against a bright sunlit background. These applications require an imager in which the pixel has a very high dynamic range (e.g. 100 dB). The pixels used in CMOS cameras (for example, the pixel shown in FIG. 2) typically have a pixel dynamic range of 70 dB or less. They are not suitable, therefore, for these high dynamic range applications. For these applications, a pixel with increased dynamic range is required.
An obvious approach to increasing the dynamic range of a pixel is to increase the value of the integration capacitor. This increases the saturation level of the pixel. Unfortunately, it also reduces the conversion gain of the pixel which reduces sensitivity of the imager and therefore the signal to noise ratio at low light levels. So the net improvement in dynamic range may be small. Increasing the size of the integration capacitor also causes the area of the pixel to increase significantly to accommodate the extra capacitance.
One technique that has been proposed for increasing the dynamic range of pixels is to use a non-linear element to compress the output of the pixel. The photodetector current may, for example, be fed into a logarithmic current to voltage converter such as a diode-connected MOS transistor. While such devices can achieve very high dynamic range, they suffer from poor sensitivity, low signal to noise ratio and exhibit high levels of fixed pattern noise.
A second technique uses conventional charge accumulation when the illumination level is low but records the ‘time to saturation’ under high levels of illumination. Once a nominal saturation level is achieved, a comparator switches to sample the voltage of an analog ramp that is supplied to every pixel. The sampled voltage provides a measure of the time instant when saturation occurred. This scheme requires a low noise analog ramp and precision components within the pixel to achieve low noise. It also suffers from high power dissipation because the precision comparator is “always on”.
A third technique uses an overflow gate to dynamically adjust the saturation level during integration. In a conventional 3-T pixel, the gate of the pre-charge device is pulsed “high” at the beginning of the integration cycle and then held “low” during the integration period. By placing a small positive control voltage on the gate of the pre-charge device during the integration period, one can effectively lower the saturation level of the pixel. During the initial portion of the integration period, the saturation level is set low. Once saturation is reached, any additional photocurrent is drained away through the pre-charge device. After a predetermined interval, the saturation level is raised. Charge can then once again accumulate until the new saturation level is reached. The saturation level is monotonically increased in steps during the integration period in such as way as to create a well defined non-linear charge to voltage relationship. This scheme enhances dynamic range at the expense of reduced signal to noise ratio due to an effectively reduced charge storage capacity.
A fourth technique proposes a reset gate to drain optical charge from the photo-detector. By selectively activating the reset gates of individual pixels, one can individually set the effective integration time of each pixel. This technique was proposed for use with CCD imagers. It could also be applied to CMOS imagers. However, control of the reset gate is external to the pixel array. It thus requires considerable external circuitry to remember the recent activity of each pixel and then a complex 2-D addressing scheme within the array to individually control the reset gate of each pixel.
It has further been suggested to employ a multiple sampling technique based on pixel level analog-to-digital (A/D) conversion. The in-pixel A/D converter uses a technique known as multi-channel bit-serial (MCBS) to convert the analog output of the pixel to a Gray code digital output. A block diagram of the A/D converter is shown in FIG. 3. It comprises a comparator, and a D-latch. This circuit generates one bit of the Gray code digital output Dout at a time. The voltage to be converted Ain is supplied to one input of the comparator. An analog ramp Aramp is supplied to the other input of the comparator. An m bit Gray coded digital ramp Dramp, whose digital value at any time t corresponds to the value of the analog ramp at time t is also provided. The ith bit of the digital output Douti is determined by supplying Drampi to the D input of the latch. Drampi is a binary digital waveform whose value at any time t is equal to the ith bit of the digital Gray code ramp. When Aramp is equal to the input value Ain, the comparator switches and stores the appropriate digital value into the latch. This process is performed for each of the m bits of the digital output. Gray code is used so as to minimize errors that would be caused by small changes in the input while generating a multi-bit digital output.
The output of the pixel is sampled by the A/D converter at a series of k exponentially increasing integration times T, 2T, 4T, . . . , 2kT. The digitized output of the A/D converter will approximately double each time until saturation is reached. Suppose that saturation is first detected when performing the jth conversion, that is, after time 2jT. The output of the pixel is then the m bit output of the A/D converter after the (j−1)th conversion (last output prior to saturation) multiplied by a scale factor 2k−j+1 to account for the reduced integration time. This scheme has the advantage that all A/D conversion is performed inside the pixel; no analog output is required. It also increases the dynamic range of the pixel by a factor of 2k while providing m bits of resolution at all levels of illumination.
With this approach, it is not necessary to output all m bits at each sampling instant. All m bits are output at the first sampling instant (at time T). Assuming the charge to voltage response of the pixel is linear, the output at time 2T will be double the output at time T. Once the scaling factor is applied, the outputs will be identical except for the least significant bit (LSB). This LSB is the one bit of added precision that has been obtained by doubling the integration time. After the first integration time, therefore, it is only necessary to output the LSB of the m bit digital output. Each subsequent sampling instant yields another bit of the digital output until saturation is reached.
There are, however, a number of problems associated with this scheme. Firstly, the accuracy of the digital readout requires that the optical illumination be constant over the integration period and that the conversion from light to charge and charge to voltage be linear. Variations in illumination and/or non-linearities in the circuit invalidate the assumption that the output at time 2T will be double the output at time T and hence the property that only the LSB of the converted output will change after the first sample. Small non-linearities, for example, can cause inconsistencies between the output bits in successive samples which can, in turn, cause (potentially large) errors in the digital readout. Secondly, the sensitivity of the pixel is limited by the resolution of the MCBS converter which is, in turn, limited by the gain-bandwidth product of the comparator and the accuracy of the analog and digital ramps. Area and power limitations within the pixel preclude the use of a high-gain, high-bandwidth comparator. Quantization noise in the converter is therefore likely to be much greater than kTC noise or the analog read-out noise that would be found in a conventional imager. This will likely limit the use of this type of pixel in low-light conditions.
Accordingly, it would be advantageous to provide a CMOS imaging sensor with a pixel design which has a large dynamic range without compromising low-light performance.